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AN INTEGRATED APPROACH TO INTELLIGENT SYSTEMS 
FINAL REPORT (JANUARY 2002) 

S. Shankar Sastry, Anil Nerode and Zohar Manna 


1. Statement of the Problem Studied 

This report presents a technical summary of the activities of the MURI during the 
period August 1996 through July 2001. To summarize, the main contributions of the 
MURI in this period have been to advance the basic state of knowledge in: 

1. Multi-modal control of "single agent" systems: modeling, analysis, control law 
synthesis and simulation 

2. Design of hierarchical control architectures for multi-agent systems that share 
a single environment 

3. Perception systems: (a) hierarchical aggregation (b) wide-area surveillance 
and (c) low-level perception 

4. Frameworks for representing and reasoning with uncertainty 

5. Soft computing and evolutionary approaches to the design of complex 
systems 

To our knowledge, we are the premier group in the country that has provided the 
analytic tools and frameworks for: 

1. A theory of analysis, verification and synthesis of control laws for hybrid 
systems involving a combination of state-based continuous state dynamical 
models and IO-based concurrency models 

2. Software tools for the simulation of hybrid multi-modal systems (SHIFT), the 
model-based verification and synthesis of hybrid control design (HyTech), 
and combined deductive-inductive verification tools (STeP) 

3. Methods for the integration of multi-agent objectives into consistent, safe 
plans with performance metrics to quantify the emergent behavior 

4. Fundamental new theory for the integration of control and sensory hierarchies 
(especially visual), with application to automatic driving and teleprescent 
environments 

5. Novel methods for segmentation of images, implementation of low-level 
vision algorithms on neural network hardware, and the use of chaotic neural 
networks for representing complex patterns 

6. Inference in dynamic probabilistic networks, first-order probabilistic modelling, 
multisensor monitoring of complex environments 

7. Provably convergent algorithms for reinforcement learning 

8. Model-based approaches to soft computing involving the simultaneous use of 
model-based approaches with fuzzy control, genetic algorithms 

1.1. Collaborative Arrangements 

The Center has developed collaborative arrangements with several other laboratories 
and other centers, such as: 

• Yamaha Motor Co., Ltd. Aeronautic Operations, Shizuoka, Japan, autonomous flight 


3 



DAAH04-96-1 -0341 


• Center for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden, 
in the areas of vision and control, multi-agent systems 

• INRIA -CNRS VERIMAG, Grenoble, France, in the area of software tools for 
verification 

• CNRS Robotics Laboratory, Toulouse, France, in the area of autonomous agents 

• Control Laboratory, Lund, Sweden, in the area of software tools for the simulation of 
hybrid systems 

• Floneywell Technology Center, Minneapolis, in the area of hybrid systems for flight 
management 

• Magnetti Marelli, Rome, Italy, and Cadence Berkeley Labs in the area of hybrid 
control design tools for automotive control and CAD tools for verification 

• University of Pennsylvania, mobile robots, visual servoing and verification 

• NASA Ames Research Center, in the area of advanced air transportation 
technologies 

• Laboratory for Computer Sciences, MIT, in the area of theory of distributed systems 

• California PATH program on platooning and multi-agent collaboration 

• Systems and Robotics Group, University of Porto, Portugal, in the area of 
coordination and control of multiple, autonomous, underwater vehicles 

• NEXTOR, FAA Center of Excellence in Air Traffic Operations, at Berkeley, MIT, 
University of Maryland and VPI in the area of coordinated decision making and 
conflict resolution 

• Intel Corporation and Xerox Corporation in the area of CAD tools for verification 

• SRI in the area of verification tools 

• Boeing, St. Louis in the area of Unmanned Aerial Vehicles 

• Northrop Grumann, Los Angeles in the area of Unmanned Aerial Vehicles 

• University of Southern California, Electrical Engineering, in the area of map building. 


2. Research Accomplishments 

2.1 Distributed Multi-Agent Control Systems Architecture 

Investigators: Shankar Sastry and Claire Tomlin 

Claire Tomlin's group at Stanford University has been working on a mathematical 
model to represent and analyze numerous aircraft taking off ande landing 
simultaneously and designing a system in which conflict resolution for multi-aircraft 
scenarios is simplified through the use of flight modes, which are segments of constant 
heading, of constant bank angle, and of constant airspeed, familiar to pilots and easily 
implementable by autopilots. The problem of controlling an aircraft to such a trajectory 
plan is a hybrid one: it involves designing both the continuous control laws for the aircraft 
within each flight mode as well as the method for sequencing the discrete modes 
together. The construction of conflict resolution maneuvers as a sequence of discrete 
modes allows improvement in computational complexity over the full continuous 
trajectory planning algorithm, and more importantly, allows one to mathematically prove 
the safety of the maneuver for multiple aircraft conflicts. Her work on air traffic control 
automation involves conflict resolution of multiple aircraft in a free-flight scenario, where 
each aircraft follows trajectories that are mathematically guaranteed to avoid collisions 
with nearby aircraft. The take off and landing of multiple aircraft involves closely-spaced 
parallel approaches. Provably safe closely-spaced parallel approach maneuvers would 
ease the congestion of airports for which parallel runways are too closely spaced to 
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allow simultaneous landings under adverse weather conditions. 

Tomlin's group has investigated hybrid dynamical systems with applications to air 
traffic management. Hybrid dynamical systems are continuous-time systems that have 
many different modes, or phases of operation. A switch from one mode to another may 
be triggered by an event that is either internal or external to the system. While the 
switches between modes have a defining logic and may be modeled by discrete-event 
systems, it has been shown that discrete-event or continuous-time tools alone are not 
adequate to model, analyze, or verify hybrid systems. The importance of a theory for 
hybrid control is its ability to provide a concise framework to analyze and design control 
algorithms and protocols for multi-agent, multi-objective systems. An excellent example 
of such a system is a partially automated air traffic management system. In such a 
system, much of the planning and control functionality for each aircraft is performed 
automatically by on-board avionics. However, the safety of such algorithms is currently 
"verified" by simulation for long periods of time with various initial conditions and inputs. 
She has proposed a method for designing hybrid control laws that can be used to 
guarantee safety of such systems. The method uses dynamic game theory, and 
generalizes a controller design method for purely discrete-event systems, and is used to 
design control laws for the automated resolution of trajectory conflicts between multiple 
aircraft, flight management system design, closely-spaced parallel approaches in 
automated air traffic control, and unmanned aerial vehicle design and control. 

Shankar Sastry's group at Berkeley designed and fabricated autonomous 
unmanned aerial vehicles (helicopter-based robots) and unmanned ground vehicles 
(four wheel drive, differential skid-steering all-terrain robots) capable of supporting 
research on a number of topics, including multi-agent hybrid systems involving sensor 
fusion, coordinated mission planning and distributed control. The unmanned aerial 
vehicles (UAVs) are responsive to requests for basic flight maneuvers such as takeoff, 
landing, hover and waypoint navigation. Each UAV consists of a base airframe and 
integrated avionics systems, that includes path planning, flight control system, navigation 
sensors and a communication module. A team of unmanned ground vehicles (UGVs) 
are equipped for similar research operations, with a base all-terrain vehicle and 
navigation system which decomposes a mission into self-localization or position 
estimation waypoinhts based on a combination of dead reckoning with periodic 
compensation using external navigation data. GPS is the primary navigation sensor. 
Other components for sensing and navigation include cameras, position encoders, 
digital compases and range-finding ultrasonic sonar transducers. The autonomous 
vehicles exchange data over a wireless network. The UAV/UGV system is inherently 
hybrid, having to combine continuous control with discrete logic. Hierarchy makes it 
possible to separate complex global tasks into a series of simpler, local ones. The 
strategy planning and probabilistic map building of the environment is done either on 
board each vehicle or at a base station, depending on the degree of information 
centralization. 

To validate the proposed architecture, pursuit-evasion game scenarios have 
been enacted in which a group of pursuers are attempting to capture another group of 
evaders within a fixed and unknown arena that may contain fixed obstacles. A 
hierarchical architecture was developed for coordination and control of autonomous 
agents performing intelligent team operations. Blue and red teams, consisting of 
multiple aerial and ground vehicles coordinate strategy through communication via a 
wireless network. Intelligent multi-agent systems address optimization, coordination, 
fault detection-tolerance, stability, with reduced control and cognition complexity, 
adaptivity to changes in task and environment, modularity and scalability to perform 
complex missions. 
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ARO MURI papers 2000-2001 

A Game Theoretic Approach to Controller Design for Hybrid Systems. Claire Tomlin, 
John Lygeros and Shankar Sastry. Proceedings of the IEEE, vol 88, no. 7, 949-969, 
2000 . 

Hierarchically Consistent Control Systems. George Pappas, Gerardo Lafferriere and 
Shankar Sastry. IEEE Transactions on Automatic Control, vol 45, no. 6, 1144-1160, 
2000 . 

Control System Design for Rotorcraft-based Unmanned Aerial Vehicles Using Tiome- 
domain System Identification, David Shim, H. Jin Kim and Shankar Sastry. IEEE Inti 
Conference on Control Applications, Anchorage, 2000 

A Flight Control System for Aerial Robots: Algorithms and Experiments, H. J. Kim, D. H. 
Shim, and S. Sastry. 15th IFAC World Congress on Automatic Control, Barcelona, 
Spain, July, 2002. (accepted for publication) 

A Flight Management System for Unmanned Aerial Vehicles with Nonlinear Model 
Predictive Tracking Control, H. J. Kim, D. H. Shim, and S. Sastry. ACC, Anchorage, AL, 
May, 2002 (accepted for publication). 

FLYING ROBOTS: Sensing, Control and Decision Making, H. J. Kim, D. H. Shim, and S. 
Sastry. IEEE International Conference on Robotics and Automation, Washington, DC, 
May, 2002. (accepted for publication) 

A Hierarchical Approach to Probabilistic Pursuit-Evasion Games with Unmanned Ground 
and Aerial Vehicles, H. J. Kim, R. Vidal, D. H. Shim, O. Shakernia, and S. Sastry. IEEE 
Conference on Decision and Control, Orlando, FL, December, 2001. 


2.2 Hierarchical Sensing and Control 

Investigators: Jitendra Malik and Shankar Sastry 

Capability of reasoning in terms of objects and their behaviors is crucial for mid 
and higher level control. This requires that we be able to group the spatiotemporal 
information into separate objects. 

Jitendra Malik's group developed a novel framework for grouping known as 
Normalized Cuts. The image is considered to be a weighted graph where the nodes are 
pixels and arcs weights denote a local measure of similarity between the two pixels. 
Grouping can then be performed by finding eigenvectors of the Normalized Laplacian of 
this graph. Brightness, color, texture, motion similarity, proximity and good continuation 
can all be encoded into this framework. We also came up with a solution to the problem 
of cue integration for image segmentation. 

Identifying particular combinations of regions as being instances of known 
objects. We introduced a new shape descriptor, the shape context, for correspondence 
recovery and shape-based object recognition. The shape context at a point captures the 
distribution over relative positions of other shape points and thus summarizes global 
shape in a rich, local descriptor. We use the shape context as a vector-valued attribute 
in a bipartite graph matching framework. Our method makes use of a relatively small 
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number of sample points selected from the set of detected edges; no special landmarks 
or keypoints are necessary. Tolerance and/or invariance to common image 
transformations are available within our framework. 

Shape contexts greatly simplify recovery of correspondences between points of 
two given shapes. Moreover, the shape context leads to a robust score for measuring 
shape similarity, once shapes are aligned. It is thus a generic method with applications 
in object recognition, image registration and point set matching. Using examples 
involving both handwritten digits and 3D objects, we illustrated its power for object 
recognition. 

The other main focus of our research effort was in developing a deeper 
understanding of multiview geometry, as applicable to the recovery of structure from 
motion. Multiview geometry conceptually, can be viewed as a geometry which studies 
the combination of any motion group (acting on certain spaces) and a perspective 
projection (properly interpreted in respective spaces). In the default case, the motion 
group is the special Euclidean group acting on the three-dimensional Euclidean space, 
and the perspective projection is simply the standard one. In the same spirit, we should 
be able to study vision under any motion groups. The traditional projective geometry 
based approach can then be interpreted as arising when the special Euclidean group is 
substituted by the general linear group for simplification. Respectively, the underlying 
space becomes a projective space rather than the Euclidean one. To take this one step 
further, if the motion group is replaced by some other ones, the underlying spaces can 
even be any non-Euclidean spaces. 

We have unified our results and understanding in multiview geometry in a framework 
which we call “a differential geometric approach". The conceptual interpretation of 
multiview geometry above is by no means the only reason for us to adopt differential 
geometry. Our work has shown that differential geometry serves well as a framework 
which provides us geometric intuition, interpretation and tools to almost every problem 
that we have studied about multiview geometry, for example: 

• The unification of discrete and continuous linear algorithms relies on a clean 
geometric characterization of the space of essential and continuous essential 
matrices. 

• The nonlinear algorithms for obtaining optimal (or suboptimal) estimates rely on 
modern optimization techniques for special classes of Riemannian manifolds. 

• The proof of geometric dependency of constraints among multiple images relies 
on a clever trick on a quotient space of a Grassmann manifold. 

• The discovery of degeneracy of Kruppa's equation relies on a new interpretation 
of Kruppa's equation as inner product invariants of certain isometry group action. 

• A classification of generic ambiguities in the problem of 2D to 3D reconstruction 
is done with respect to every Lie subgroup of the special Euclidean group. 

Another obvious reason is that differential geometry has already been widely adopted in 
the study of linear/nonlinear systems and modern robotics. A theory of multiview 
geometry based on such a language will certainly be more accessible to people in these 
communities. It may provide a more unified framework for the study of vision-based 
control or vision in presence of multiple kinematic chains. 

Over the past years of the MURI Shankar Sastry's group has performed 
numerous computer vision experiments in robotics and control applications for 
autonomous navigation, obstacle avoidance, object recognition and manipulation, 3D 
map building. In all these areas an important question is how to recover geometric and 
dynamic information from the scene being observed. 

One of the experiments using the Berkeley robotic helicopters is the landing of an 
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autonomous unmanned aerial vehicle using computer vision as a feedback sensor in a 
control loop. The vision problem addressed in this autonomous landing experiment is a 
special case of the classic ego-motion estimation problem since all feature points lie on 
a planar surface (the landing platform). The group studied together the discrete and 
differential versions of the ego-motion estimation, in order to obtain both position and 
velocity of the unmanned aerial vehicle relative to the landing platform, developed a 
unified geometric framework and a new estimation scheme for solving the differential 
case, showing how the obtained algorithms enable the vision sensor to be placed in the 
feedback loop as a state observer for landing control. These algorithms are linear, 
numerically robust, and computationally inexpensive hence suitable for real-time 
implementation. The results of landing experiments present a thorough performance 
evaluation of the motion estimation algorithms under varying levels of image 
measurement noise, altitudes of cameras above the landing platform, and different 
camera motions relative to the landing platform 

Sastry’s group also studied the classic problem in computer vision, “the structure 
from motion problem” as a constrained nonlinear least squares problem which minimizes 
the so-called reprojection error subject to all constraints among multiple images. By 
converting this constrained optimization problem to an unconstrained one, multilinear 
constraints, when used for motion and structure estimation need to be properly 
normalized, which makes them no longer tensor constraints. It was demonstrated by 
using bilinear epipolar constraints and shown how they give rise to a multiview version of 
the (crossed) normalized epipolar-constraint of two views. Such a (crossed normalized 
epipolar constraint serves as an optimal objective function for motion (and structure) 
estimation. This objective function further reveals certain non-trivial relationships 
between geometric and algebraic dependency among multilinear constraints, that is, 
even rectilinear motion can be correctly estimated by normalized epipolar constraint 
only, hence trifocal tensors are not really necessary. The proposed algorithm was 
demonstrated in simulation and experiments. 

ARO MURI papers 2000-2001 

Landing an Unmanned Air Vehicle: Vision Based Motion Estimation and Nonlinear 
Control, Omid Shakernia, Yi Ma, John Koo and Shankar Sastry. Asian Journal of 
Control, vol 1, no 3, 128-145, 1999. 

Optimal Motion Estimation from Multiple Images by Normalized Epipolar Constraint, Y. 
Ma, R. Vidal, S. Hsu and S. Sastry. Communication in Information and Systems, vol 1, 
no. 1, pp 51-75, 2001. 

Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation, Yi 
Ma, Jana Kosecka and Shankar Sastry. International Journal of Computer Vision, vol 
44, no 3, 219-249, 2001. 

Linear Differential Algorithm for Motion Recovery: A Geometric Approach, Y. Ma, J, 
Kosecka, S. Sastry. International Journal of Computer Vision, vol 36, no 1,71-89, 2000. 


2.3 Specification and Verification of Multi-Agent Hybrid Systems 

Investigator: Zohar Manna 


Since 1994, Manna's group has been developing STeP, the Stanford Temporal 
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Prover, an integrated toolset that supports the computer-aided formal verification of 
reactive systems based on temporal specifications. STeP combines model checking with 
deductive methods to allow the verification of a broad class of systems, including 
parameterized (N-component, for arbitrary N) systems, and systems with infinite data 
domains. Under the MURI project STeP has been extended to support the deductive 
verification of hybrid systems, and to our knowledge, it is the first dedicated verification 
system with built-in support for the deductive verification of such systems. 

STeP has been extended with modular and abstraction-based techniques for the 
study of hybrid and real-time systems, including the diagram-based techniques that have 
proved effective for the verification of reactive systems. In the past funding year, a large 
part of STeP has been reimplemented in Java, with the objective to create a more open 
architecture that allows integration of tools for symbolic and numerical math, which are 
necessary tools for analyzing non-trivial hybrid systems. 

To model hybrid systems, we use phase transition systems, which contain both 
discrete transitions and continuous activities that describe the evolution of continuous 
variables, using differential equations and or differential inclusions. Temporal properties 
of hybrid systems are verified by verification rules and verification diagrams. Verification 
rules are used to reduce temporal properties of systems to first-order verification 
conditions. Verification diagrams provide a visual language for guiding, organizing and 
displaying proofs. Verification diagrams have been extended to support modular 
reasoning and to prove nonzenoness and receptiveness. A diagram is constructed to 
analyze the behavior of each system module; the diagrams for the different modules can 
then be automatically combined into the correctness proof for the global system. In 
addition, techniques for automatic invariant generation of hybrid system have been 
developed. 

These techniques can be used to aid with the generation of the appropriate 
verification diagrams; in some cases, the techniques are sufficiently powerful to lead to 
the automatic proof of the specification. To aid with the analysis of verification diagrams, 
and in particular with the discharge of the verification conditions associated with the 
diagrams, we have developed a general framework for combining decision procedures. 
This framework enables to harness the power of independently-developed procedures 
towards the joint goal of proving the validity of verification conditions. This results in a 
modular, expandable architecture of the theorem prover used for deductive verification, 
which enabled the verification of complex systems with a minimum of user intervention. 
The following were the project headings supported by the MURI: 

• Verification of performance and reliability of real-time systems 

• Game-based specification and verification techniques for multi-agent systems 

• Visual, deductive methods for the modular verification of hybrid systems 

• Use of deductive techniques jointly with model checking 

• Logics and automata-based methods for the specification and verification of real¬ 
time and hybrid systems 

2.4. Architectures for Multi-Agent Hybrid Systems 

Investigator: Anil Nerode 

Nerode’s group has constructed a natural common semantics which meets 
stability requirements not naturally guaranteed by correctness in existing languages, and 
is explained in terms of the Kohn-Nerode methodology. The goal of the Kohn-Nerode 
hybrid distributed autonomous control architecture is to establish the mathematics and 
computer science of extracting and checking correctness of digital control programs 
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which claim to enforce performance specifications for such systems. We regard as a 
key issue assuring that an extracted control program still meets the program 
specification even if the parameters of the plants controlled and the communication links 
are not exactly what we think they are; and that we should be able to compute how 
much error in parameters still keeps us within the program specification. This is a weak 
necessary condition form of stability. We believe that this has to be part of correctness, 
needed to write adequate design requirements for sensors, effectors, communication 
lines, and plant. 

If, as many workers have, one turns the whole hybrid system into an automaton 
by digitizing sensor, effector, plant, and communication line states, and then extracts a 
digital control program by automata theory, one has no assurance that with small 
changes in the parameters of the system the program will still meet performance 
specifications. The same thing often happens if one simply writes every event driven 
decision in an event driven language as if every value is perfectly known. Further, when 
one composes get estimates in general for the tolerable error of the composition from 
that for the parts. If one treats this directly as a numerical analysis problem without 
discretizing, there is no high-level language expressing this. Many performance 
specifications have nested quantifier specifications. These arise with liveness and 
termination, and whenever one sort of firing of an event always or never causes another, 
etc. Stability under small changes of parameters should be proved for these nested 
statements as well. 

We believe we have a satisfactory answer with new semantics tuned to hybrid 
systems such that programs correct with respect to this semantics have the kind of 
stability referred to above. This is a new semantics for all the languages for hybrid 
systems. This common semantics for all hybrid languages will allow them to be "mixed 
and matched". Programs in each language correct in this new semantics can be readily 
composed, so that calling subroutines compiled in one language into another is just fine. 
The crudest form of the semantics keeps track of tolerance to errors. They are not 
named in the language itself. We confine ourselves to describing this semantics. 

Hybrid control programs that have been declared correct by current language 
semantics of current hybrid languages may not be correct in the new semantics. With 
some exceptions such as some of Nancy Lynch's work, or piecewise linear plants, they 
usually have to be reworked using mathematical analysis and rewritten to get programs 
correct in the new semantics. For the new semantics declares that those programs that 
cease to obey the performance specification for arbitrarily small changes in some 
physical parameters are incorrect. This semantics sharpens present correctness notions 
for hybrid systems to give a more engineeringly meaningful notion where correctness 
involves the analysis of behavior of the differential or variational or other equations 
obeyed by the physical plants being controlled. We have believe that appropriate Kohn- 
Nerode algorithms for extracting control programs are correct in the new semantics, 
assuring the form of stability mentioned above. All computer language semantics can be 
viewed as Scott semantics on an appropriate CPO, just as all mathematics can be 
viewed as category theory on an appropriate category. That is true here, but as an 
insight it does not give value; that is, there are no deep theorems to apply. 

We explain briefly the new semantics, using Prolog as a control program 
language. We omit other languages because they take some effort to explain. In the 
Kohn-Nerode setup a Prolog program is the control program; the input state of the plant 
fires an atomic statement. The Prolog program computes and terminates. The answer 
substitution gives the next control law to use. The requirement that an arbitrarily close 
plant states fire the same input letter plus the finiteness of the input alphabet says that 
the topology generated by the open sets that fires the input letters is a finite distributive 
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lattice of sets. It generates a finite Boolean algebra. The atoms are a partition of the 
plant sets. The control automaton cannot distinguish between plant states if they are in 
the same partition set. 

We take it that there are two topologies involved on the plant state space: the 
usual plant state topology, and the finite topology just mentioned, which is a finite 
topology representing what the control program can distinguish about the plant states 
through the sensors. It is natural to say that each Prolog atomic statement denotes the 
open set of plant states that fire it. This open set is a subset of the original plant state 
space with its original topology and is also in the plant state space small topology. We 
now apply the Tarski-Rasiowa-Sikorski topological semantics for predicate logic of which 
Prolog is a part; negation is omitted because it is definable from implication and falsity. 
Every atomic statement has been assigned a denotation which is an open set in the 
plant topological space. Define by induction on the length of formulas the denotation of 
every statement: “and” goes to intersection, “or” to union, “A implies B” to largest open 
set containing A and disjoint from B, “T” to the whole space, “F” to the empty open set, 
existential quantifier to the union over instances, universal quantifier to the interior of 
intersection of instances. In case there are constants naming every partition set and the 
predicates are about these partition sets, as is the case with our Prolog programs, the 
universal quantifier reduces to a finite union, and we are dealing with a finite model 
whose domain consists of the partition sets. A correct statement is one denoting the 
whole space. Correctness in our sense refers to a specification being satisfied by the 
Prolog program in the sense of this topology. The statements valid in all such topological 
models, as the topological space varies, Rasiowa-Sikorski proved are those that are 
intuitionistically valid. This implies that the rules of inference to be used are the 
intuitionistic ones only, and that we are talking about intuitionistic validity in a finite model 
associated with a finite topology when we prove the correctness in our sense of a Prolog 
program. Programs with input and output letters should be composed so that if an 
output atomic statement is the input to another such program, the open set denoting the 
output atomic statement should be a subset of the open set denoting the input atomic 
statement. 

This gives a hint as to how the same ideas will apply to all other proposed 
formalisms. We are suggesting that only intuitionistically correct deductions be used (as 
in Prolog), and that specifications are read the the same way, and can be in full 
intuitionistic logic. We do not have space to explain why this interpretation guarantees 
stability in the sense we mentioned of performance specifications with alternating 
quantifiers in their statement, but it does. We cannot go further here. 

Nerode’s work on hybrid systems theory and applications was devoted to further 
development of the Kohn-Nerode theory of multi-agent hybrid control systems based on 
the calculus of variations, automata theory, and differential geometry as a basis for 
intelligent systems. The aim of this theory is to find algorithms for extracting digital 
control programs from performance specifications and simulation models for complex 
interacting heterogeneous systems, or hybrid systems. Such a system consists of both 
continuous plants and digital programs. In a large military problem, the plants may be 
military assets such as tanks and planes, the digital programs may be Al command and 
control assistants, the simulation models may be for distributed simulation. At a more 
pedestrian level, the plants may be power stations and power users, the digital programs 
may be those that route power through distribution centers. Synchronization of 
multimedia applications. Distributed simulation problems are of the same genre. 

Nerode also explored non-monotone logics for Al and hybrid systems which 
involved an exploration of logics and algorithms for intelligent systems which deal with 
how to change "belief systems" when facts intervene to show that the beliefs cannot be 
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valid and must be replaced by new "belief systems". Suppose that a system under digital 
control is observed to fail to meet its performance specification, which it would meet if 
the model of the system used in designing the control system was correct. How does 
one revise the model using this new information, and derive a new control system 
controlling the new structurally changed model? This is within the area of non-monotonic 
reasoning. A typical military problem of this type is how to replan, redeploy, reallocate 
resources as friendly assets are destroyed and unexpected new enemy assets arrive in 
the field. Also of this genre is making sure that when part of a complex system under 
control fails, that the control is changed to fit what is known about the structural model of 
what is left. Power system failures due to storms, or due to failures of large generators, 
are of this type. Algorithm development in this area is very taxing. 

Jennifer Davoren, a postdoctoral researcher in Nerode’s group, formulated and 
justified the view that the modal mu calculus, among all current formal systems, is best 
suited to model and verify hybrid systems. She showed by example that its natural proof 
procedures correspond very closely to those used in informal verification, a substantial 
advantage over temporal logic formulations. She investigated hybrid systems as 
heterogenous dynamical systems characterized by interacting continuous and discrete 
dynamics. The high-profile and safety-critical nature of the application areas, such as air 
traffic control and automated manufacturing and chemical process control, has fostered 
a large and growing body of work on formal methods for hybrid systems: mathematical 
logics, computational models and methods, and computer-aided reasoning tools 
supporting the formal specification and verification of performance requirements for 
hybrid systems, and the design and synthesis of control programs for hybrid systems 
that are provably correct with respect to formal specifications. Their work on the MURI 
program continues to provide original contributions to the use of logics and formal 
methods in the analysis of hybrid systems, specifically in the area of hybrid controller 
synthesis problems. Recent work formulates and robustly solved a quite general class 
of hybrid controller synthesis problems, focusing on the switching control mechanism of 
a hybrid automaton (via guard and mode invariant sets). The robustness result is with 
respect to variations in the right hand sides of the differential equations that depend 
continuously on a parameter. We present a novel methodology for controller design and 
synthesis which uses modal logic as a formalism for reasoning about sets of plant 
states, and various operators on sets arising from the differential equations and from 
metric tolerance relations on the state space. 


2.5. Intelligent Agents for Complex, Uncertain Environments 

Investigators: Stuart Russell and Daphne Koller 

Russell's group advanced the state of the art by developing new methods for 
learning in the presence of incomplete data (e.g., data with missing values and hidden 
unobservable variables)-major computational overhead in learning. To facilitate 
learning in complex domains, inference algorithms that are especially tuned for the 
learning task A novel algorithm for probabilistic matching was developed and applied it 
for matching vehicles in image sequences from traffic surveillance cameras that view 
different segments of the same highway. New hierarchical decomposition techniques 
were developed that allow to perform decision-theoretic planning and reinforcement 
learning in large domains that cannot be handled by standard methods. 

Koller's group made progress in devising frameworks that support the 
hierarchical decomposition of complex stochastic systems into multiple interacting 
processes. These include a stochastic extension of interacting finite automata for 
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representing stochastic processes and a stochastic extension to a functional 
programming language for representing distributions over regular structures such as 
parse trees These works lay the foundations for combining methods from the 
probabilistic network literature to hybrid systems. In another line of research, Koller's 
group made interesting progress in its work on learning, inference, and planning. Focus 
was on learning Bayesian networks, intelligent decision modeling, and on planning in 
Markov decision processes (MDPs), specifically, the modeling of decision behavior. This 
task is fundamental to many applications at the forefront of Al, for example, in the design 
of multi-agent systems or for human-computer interaction. Its solution entails 
representing an agent's behavior in terms of a suitable class of utility functions and 
identifying a concrete element of this class using one of various techniques. In 
particular, the group explored an approach where the utilities can be viewed as random 
variables whose density can be estimated using historical interactions. This density can 
be used as a prior, which is then conditioned based on observations of the agent's 
behavior. The group developed an algorithm for performing utility elicitation in a way 
that reveals the agent's utility function using a small number of elicitation questions. 

A topic that is closely related to utility solicitation is the identification of optimal 
decision rules based on the agent's utility function. This objective becomes particularly 
challenging if there are dynamic interdependencies between the decisions at different 
points in time. This situation can be modeled using the MDP framework. The group's 
work led to several new advances in this area. In particular, it extended its vision of 
simplifying probabilistic computations by exploiting problem structure. A key 
development was a new algorithm that approximates the solution to a complex factored 
MDP using approximate policy iteration. This algorithm leads to exponential 
computational savings by comparison to the exact solution of the factored MDP. 

An ongoing theme of the group's work is the learning and inference in Bayesian 
networks. The members of the group have focused on several aspects: First, they 
investigated algorithms to learn model structure in a Bayesian framework. Second, they 
extended their existing work for parameter estimation in Bayesian networks to include 
“active learning". Active learning is crucial in scenarios where we can choose between 
different data items for learning at each time step. It is particularly important in cases 
where obtaining data is particularly expensive. Active learning of model structure is a 
completely novel approach that was found to be extremely useful for dealing with very 
large data sets. 

A special class of Bayesian networks that is suited for many real problems are 
Dynamic Bayesian Networks (DBNs). DBNs have been promoted as a powerful 
framework for the modeling of dynamic systems in earlier work, with a focus on efficient 
algorithms to exploit problem structure. In the reporting period the group extended this 
theoretical framework so as to allow also for the modeling of fault detection and 
diagnosis in dynamic systems. An important aspect of this work is that it focuses on 
DBNs which may contain both discrete and continuous random nodes. The modeling of 
these hybrid systems is computationally particularly challenging. At the same time the 
solution of this problem is particularly important because most real-world systems are 
hybrid by nature. The newly algorithm lead to drastic performance improvements in 
various example problems. 

The extraction of single features that serve to approximate the state of the 
system in a compact manner. While feature selection is a relatively standard approach to 
deal with pattern recognition tasks, it has been used essentially as a heuristic in the 
context of MDPs. In this new work the group established for the first time a direct formal 
link between the performance loss due to the approximation and the information about 
the system state contained in the features. Initial experiments demonstrate that a very 


13 



DAAH04-96-1 -0341 


small number of features can be sufficient to approximate the exact solution of an MDP 
well. This direction of research may lead to a breakthrough in the practical applicability of 
Markov Decision Problems and Partially Observable Markov Decision Problems 

Jordan's group developed a methodology for approximate probabilistic inference 
known as "variational inference." The basic idea is that the symbolic complexity inherent 
in expanding sums of products in probabilistic computations is not necessarily reflective 
of the numerical complexity of the final result. In particular, laws of large numbers come 
to the rescue, and many sums do not need to be performed explicitly because their 
result is predictable. The goal of their work was to design algorithms that exploit this 
idea in the graphical models setting. They developed a number of such algorithms, 
using ideas from statistical physics, convex analysis and large deviation theory. The 
result was a suite of deterministic algorithms that provide anytime bounds on marginal 
and conditional probabilities of interest. 

Jordan's group developed a variational inference algorithm for diagnosis in a 
large-scale graphical model. This model—the QMR-DT medical database—involves 
approximately 5000 nodes and is one of the largest probabilistic databases in existence. 
The variational algorithm provides accurate estimates of diagnostic probabilities for 
QMR-DT in under a second of computer time. By way of comparison, it is estimated that 
exact probablistic inference algorithms for QMR-DT would require 50 years to run. It is 
also known that sampling algorithms (Markov chain Monte Carlo algorithms) are slow 
and give unreliable results for QMR-DT. 

ARO MURI papers 2000-2001 

Active Learning for Parameter Estimation in Bayesian Networks, Simon Tong and 
Daphne Koller. Proceedings of the Seventeenth Annual Joint Conference on 
Intelligence, 863-869, 2001 

Support Vector Macine Active Learning with Applications to Text Classification, Simon 
Tong and Daphne Koller. Proceedings of the Sixteenth Annual Conference on 
Uncertainty in Artifical Intelligence (UAI-00), 999-1006, 2000 

Utilities as Random Variables: Density Estimation and Structure Discovery. Urszula 
Chajewska and Daphne Koller. Proceedings of the Sixteenth Annual Conference on 
Uncertainty in Artifical Intelligence (UAI-00), 63-71,2000 

Policy Iteration for Factored MDPs, Daphne Koller and Ronald Parr, Proceedings of the 
Sixteenth Annual Conference on Uncertainty in Artifical Intelligence (UAI-00), 326-334, 
2000 

Being Baysian about Network Structure, Nir Friedman and Daphne Koller, Proceedings 
of the Sixteenth Annual Conference on Uncertainty in Al (UAI-00), 201-210, 2000 

Making Rational Decisions using Adaptive Utility Elicitation, Urzula Chajewska, Daphne 
Koller and Ronald Parr. Proceedings of the Seventeenth National Conference on 
Artificial Intelligence (AAAI-00), 363-369, 2000 

Restricted Bayes Optimal Classifiers, Simon Tong and Daphne Koller. Proceedings of 
the Seventeenth National Conference on Artificial Intelligence (AAAI-00), 658-664, 2000 

Bayesian Fault Detection and Diagnosis in Dynamic Systems. Uri Lerner, Ronald Parr, 
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Daphne Koller and Gautam Biswas. Proceedings of the Seventeenth National 
Conference on Artificial Intelligence (AAAI-00), 531-537, 2000 

Being Bayesian About Network Structure. Nir Friedman and Daphne Koller. 
Proceedings of the Sixteenth Annual Conference on Uncertainty in Artifical Intelligence 
(UAI-00), 201-210, 2000 


2.6 Soft Computing and Neural Networks 

Investigators: Walter J. Freeman and Lotfi A. Zadeh 

Walter Freeman’s MURI research was devoted to measurement and description of 
electroencephalographic (EEG) and unit activity of cortex that occurs during goal- 
directed behavior. Behaviorally relevant information is carried by spatial patterns of 
amplitude modulation of gamma waves (20-80 Hz in rabbits, 35-60 Hz in cats), which 
are triggered in the cortex by input from receptors and relays in frames at theta 
frequencies (2-7 Hz). Behavioral testing has shown that amplitude patterns of gamma 
activity are invariant with respect to learned odor stimuli, but change with context and 
reinforcement under conditioning. The same algorithms hold for olfactory, visual, 
auditory and somatic cortexes. These spatial patterns are shaped by inputs from the 
limbic system in the form of corollary discharges, as the basis for attention and 
expectancy, and by the history of experience with stimuli that is embedded in the cortical 
neuropil by synaptic modifications due to learning. The patterns manifest not the 
features of stimuli, but the meaning of the stimuli for the animals. Freeman has modeled 
the dynamics of the cortex by networks of nonlinear differential equations. The solutions 
to these equations show the existence of equilibrium, limit cycle and chaotic attractors, 
which we display with interactive graphics. The model performs with outstanding efficacy 
the basic functions of sensory cortex: abstraction, generalization, and classification. 

Existing methods of complexity research are capable of describing certain specifics 
of biosystems over a given narrow range of parameters. In his MURI project a new 
approach to chaos research was used that has the potential of characterizing biological 
complexity. This study is biologically motivated and solidly based in the biodynamics of 
higher brain function. At the same time it outlines a deep mathematical analysis into the 
foundations of chaos theory and dynamical systems. Freeman's biocomplexity model 
has the following features: 

• It is high-dimensional, but the dimensionality is not rigid, rather it 
changes dynamically; 

• It is not autonomous and continuously interacts with and 
communicates with individual environments that are selected by 
the model from the infinitely complex world; as a result, it is 
adaptive and modifies its internal organization in response to 
environmental factors by changing them to meet its own goals; 

• It is a distributed object that evolves both in space and time toward 
goals that it is continually re-shaping in the light of cumulative 
experience stored in memory; 

• It is driven and stabilized by noise of internal origin through self¬ 
organizing dynamics. 

Of special interest are input-induced and noise-generated, or spontaneous state- 
transitions and the stability properties of the emergent systems over certain spatial and 
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temporal scales. Adaptation and learning are key components of our model. He applied 
his new mathematical tools for industrial and biological information processing to data 
sets from our laboratory, using landscapes of chaotic attractors. The sensitivity to initial 
conditions might seem to hold promise for signal detection, owing to an implied capacity 
for distinguishing small differences in patterns. However, this sensitivity is incompatible 
with pattern classification, because it amplifies irrelevant differences in incomplete 
patterns belonging to the same class, and it renders the network easily corrupted by 
noise. We developed a theory of stochastic chaos, in which aperiodic outputs with l/f2 
spectra are formed by the interaction of globally connected nodes that individually are 
governed by point attractors under perturbation by continuous white noise. The 
interaction leads to a high-dimensional global chaotic network that is stabilized by 
additive noise modeled on biological noise sources. Systematic parameterization of the 
interaction strengths corresponding to synaptic gains around nodes representing 
excitatory and inhibitory neuron populations enabled the formation of a robust chaotic 
attractor landscape. Reinforcement learning from examples of patterns to be classified 
using habituation, association and re-normalization creates lower dimensional local 
basins in the global attractor landscape with one basin for each class. Thereafter, 
presentation of incomplete examples of a test pattern leads to confinement of the Kill 
network in the basin corresponding to that pattern, which constitutes many-to-one 
generalization. The capture after learning is expressed by a stereotypic spatial pattern 
of amplitude modulation of a chaotic carrier wave. Sensitivity to initial conditions is not 
longer an issue. Scaling of the additive noise as a parameter optimizes the classification 
of data sets in a manner that is comparable to stochastic resonance. The local basins 
constitute dynamical memories that solve difficult problems in classifying data sets that 
are not linearly separable. New local basins can be added quickly from very few 
examples without loss of existing basins. The attractor landscape enables the Kill set to 
provide an interface between noisy unconstrained environments and conventional 
pattern classifiers. Examples were shown of its robust performance, including fault 
detection in small machine parts and the classification of spatiotemporal EEG patterns 
from rabbits trained to discriminate olfactory stimuli. 

ARO MURI papers 2000-2001 

Characteristics of the synchronization of brain activity imposed by finite conduction 
velocities of axons. Walter Freeman. International Journal of Bifurcation and Chaos vol 
10, 2307-2322, 2000 

A neurobiological interpretation of semiotics: meaning, representation and information. 
Walter Freeman. Information Sciences, vol 124, 93-102, 2000. 

Mesoscopic neurodynamics. From neuron to brain. Walter Freeman. Journal of 
Physiology (Paris) 94: 303-322, 2000 

The origin of microscopic noise in biological brains and its role in stabilizing mesoscopic 
chaotic dynamics. Systemics, Cybernetics and Informatics (SCI-00), 2000 

Biocomplexity: Adaptive behavior in complex stochastic dynamical systems. Walter 
Freeman, Robert Kozma and Paul Werbos. BioSystems, vol 59, 109-123, 2001 

Chaotic resonance: Methods and applications for robust classification of noisy and 
variable patterns. R. Kozma and W.J. Freeman. International Journal of Bifurcation and 
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Chaos vol 10, 2307-2322, 2001 

3. Experimental Testbeds 

3.1 Berkeley Aerobot Project: Vision-based Navigation System 

Implementation and Results. The following equipment is integrated on our Yamaha 
R-50 based unmanned helicopters. A similar setup is used for the Pioneer all-terrain 
ground robots. 

Navigation computer: 

Pentium 233MHz Ampro Littleboard running QNX real-time O- 

responsible for low level flight control 

NovAtel MillenRT2 GPS system, 2cm accuracy 

Boeing DQI-NP INS/GPS integration system 

Vision computer: 

Pentium 233MHz Ampro Littleboard running Linux OS-responsible 
for vision algorithms and camera control 
Sony EVI-D30 Pan/Tilt/Zoom (PTZ) camera 
Imagenation PXC200 framegrabber 
2 SanDisk 220MB PCMCIA hard disks 

Lucent Technologies’ WaveLAN wireless ethernet card for each 
computer 

Our vision computer communicates with the navigation computer over a 115200 
bps RS232 serial link. Currently, we only use that link to gather INS/GPS state 
information from the navigation system for comparison with the vision-based state 
estimates. We communicate with the PTZ camera over a 9600 bps RS232 serial link to 
send pan-tilt commands and receive the current pan-tilt state. We capture grayscale 
images from the PXC200 at a resolution of 320x240 pixels at 30 frames per second 
asynchronous to program execution through memory-mapped 10 and system signals. 
To reduce the cost of thresholding, we estimate the image histogram by considering 
one-seventh of the total image pixels. Furthermore, we use a multi-resolution approach 
in our segmentation algorithm to improve processing time, effectively performing 
segmentation on a 160x120 image. We then calculate the edges of the interior squares 
of the landing target from the original 320x240 image. We develop and debug our vision 
algorithms using MATLAB then translate them by hand to C++ for execution on the 
vision computer. LAPACK (Linear Algebra PACKage) and BLAS (Basic Linear Algebra 
Subprograms) are used for matrix operations such as multiplication, SVD, eigenvalue 
decomposition, and Gaussian elimination. Our remaining code uses a flexible object- 
oriented program structure written in C++. In particular, we developed a class 
specifically for communicating with the Sony PTZ camera. We also wrote abstract 
communication classes to allow for transparent communication through TCP/IP sockets, 
serial 10, file 10, or standard 10. We found this level of 10 abstraction reduced costs in 
developing communication layers with the navigation computer and ground station. 

Our ground station displays the current state of the vision system through a Java- 
based GUI. In addition to the current vision-based state estimates, we bit-pack the 
thresholded 160x120 binary image for display on the ground station. We find that this 
real-time view from the PTZ camera is invaluable for testing by not only giving feedback 
regarding the state of the helicopter, but by also showing the quality of thresholding 


17 



DAAH04-96-1 -0341 


algorithm and the pan/tilt control. In practice, we find that while the pan/tilt control 
performs as well as can be expected, there is still room for improvement in our selection 
of the threshold. We also display the current labeled feature point estimates to gauge 
the quality of the corner detection and labeling algorithms. In practice, we find that while 
our corner detection algorithm produces somewhat noisy estimates, our corner labeling 
algorithm is extremely robust. 

Flight Test Results. For the flight test, we saved every fifth grayscale image for 
data logging. As it turns out, the Linux (kernel 2.2.12) write caching system is less than 
optimal for this type of load. As a result, the system averaged a 1.2 second suspension 
of all processing every 10 seconds while the write cache was flushed. Naturally, this 
delay is also a function of the PCMCIA hard disk used for storing the image sequence. 
However, in a real flight, the operating system would not have the burden of logging this 
data, and a consistent 30 frames per second is otherwise sustainable. 

Figure 7 shows the results of our flight test with the UAV, comparing the output of 
the vision-based state estimation algorithm with the INS/GPS measurements accurate to 
2 cm. All plots show the state of the RUAV with respect to the landing pad. The vision 
estimates are more noisy, but otherwise follow the INS/GPS measurements. Also, 
errors in the internal and external camera calibration parameters marginally affect some 
of the estimates-the x-position and z-rotation, in particular. The delay caused by the 
Linux write caching algorithm is noticeable by the synchronized gaps in data in each 
plot.Overall, the vision-based state estimates are accurate enough to be used in a high- 
level controller for landing in a hierarchical flight vehicle management system. 


3.2 Berkeley Telesurgical Workstation 

The telesurgical workstation testbed project had a dual focus. One focus has 
been the testing and evaluation of the bimanual system by surgeons at UCSF. The 
other focus has been continued development on the unimanual system. Some of the 
project’s accomplishments: 

Results from Tests at UCSF Medical Center 

UCSF Medical Center surgeons evaluated the bimanual robot. UCSF Medical 
Center surgeons have now completed a round of evaluation on the bimanual surgical 
robot system. This evaluation has included both formal testing, such as timed trials, and 
informal feedback as well (e.g. see next item). 

Evaluated modified gripper shape. We reported last year that our existing needle 
gripper (left frame below) was best for many minimally invasive tasks, and that we would 
soon evaluate a gripper with curved jaws (right frame) for other tasks. However, in the 
evaluation by UCSF Medical Center surgeons, we found the new jaws to be so effective 
that there was no longer any need for the older straight ones. UCSF surgeons found the 
new jaws’ curvature, rounded edges, and added length enhanced their ability to suture 
and to tie knots with the robot. 

Counterbalanced the surgeon’s control sticks. To reduce fatigue and enhance 
the surgeons’ dexterity with the robot, the bimanual system’s master control station was 
upgraded to include active counterbalancing of the control sticks. Although there is still 
no force fed back from the surgery site itself, at least the distracting weight of the control 
sticks has been eliminated, allowing more of the surgeon’s attention to be focused on 
the feedback available visually. Besides comfort, dexterity, and reduced fatigue, an 
added advantage of the counterbalancing is that now the control sticks not stay put 
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where the surgeon lets go of them. 

This counterbalancing has been achieved using the motors in the Phantom designed 
for force feedback. This continuous force application by the motors sometimes 
overheats them, requiring shutdown of the system. We plan to alleviate this by adding a 
metal counterweight. This will reduce the required motor torque by a third or more, 
eliminating the threat of overheating. 

Software/Hardware Implementations 

Software development specified for the redundant encoders. Background: Each 
robot arm incorporates a gross position stage driven by three linear actuators. Each 
linear actuator has an encoder to permit closed loop control. In order to prevent the 
encoders on the linear actuators from acting as single points of failure, we have added 
redundant encoders. Furthermore, we have located these new encoders where they 
monitor the gross stage’s 3 degrees of freedom in a different way from the original 
encoders on the 3 linear actuators. By doing this, they not only verify the readings of the 
original encoders, they simultaneously monitor themselves and the validity of their 
associated kinematics calculations. However, these calculations require iterative solving 
of non-linear equations. The software challenge has been to fit iterative calculation into 
a real-time context. 

This software development plan for the redundant encoders will unfold in three 
stages. The first stage, already complete, is a thorough mechanical survey of each 
robot arm. 

The second stage, about to begin, is a detailed walk through of the existing 
kinematics software (using the original encoders). Quantitative comparisons will be 
made between values in the code and those found in the survey. As a checksum on 
these individual comparisons, each robot arm will be operated with special attention paid 
to the exact position it achieves as compared to its commanded position. To do this 
check properly, we will need to replace the current, approximate, reverse (and forward) 
kinematics with the exact solutions. However, we will not yet have a satisfactory way to 
calculate one of these (the forward kinematics) in real-time. (It will be developed in 
stage three.) However, once teleoperation is running, we need only the inverse 
kinematics. Thus, we will begin teleoperation with the existing approximate kinematics, 
and then shift-on-the-fly into exact kinematics. Observing the millimeter-scale 
movements prompted by said shifting ought to be insightful as well, telling us much 
about the magnitude and spatial shape of the difference between our exact and 
approximate solutions. 

Up to now these millimeter-size systematic (or smoothly varying) deviations have not 
affected the robot’s performance. However, to utilize redundant encoders they must be 
tracked down. 

The third stage will be to choose a method of performing exact forward kinematics in 
real-time. While not strictly necessary during teleoperation, it will be needed in order to 
utilize the redundant encoders. It will also be needed to smoothly transition into holding 
before teleoperation. This choice will be informed by observations made during stage 
two. Most likely we will choose Newton’s method. A special modification may be 
needed for the case of transitioning from idle mode to holding or teleoperate mode. In 
this special case, the Newton’s method routine might be given the power to hold of 
transition of the state machine out of idle until it converges on a solution. Once the robot 
is teleoperating, the previous position ought to provide an excellent starting point, 
permitting Newton’s method to converge in 1 to 2 iterations. 

Modify the robot software to improve motor coordination. The robot software 
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has been modified to produce smoother motion without requiring any increase in data 
bandwidth between the surgeon’s control station and the robot. This has been 
accomplished by computing an interpolated path between successive position and 
orientation coordinate sets sent by the surgeon’s control sticks to the robot. 

Last year we stated an intention to also handle for the possibility of an obstruction in 
the robot’s path by ensuring that it follows a computed path. This was intended to avoid 
going too far off of a straight-line path from one 6 D.O.F. coordinate to the next. This 
concern has turned out to be an artifact of the (no longer used) kinematics software’s 
singularity. Current software never attempts to drive the robot a large distance in a 
single time-step. Path computation is no longer helpful. Thus, this task has thus been 
terminated. 

Improve position range-checking in the robot software. To prevent unexpected 
further motion in case the robot encounters unexpected resistance (including that 
caused by a malfunction), we had planned to modify the software so as to halt robot 
motion (including applying the brakes) if excessive deviation occurs between the 
commanded and sensed robot positions. Further motion will only be allowed re-entering 
teleoperation. This is so similar to the redundant encoder checking that this task will 
now be combined into that one. 

Minimize need for surgeon & operator to interact with a computer interfaces. 

Keyboard-only interface on the surgeon’s control station has continued to work well. It is 
now possible to start and even restart (without rebooting) the surgeon’s control station 
without the need for a monitor and without having to use the mouse. We have delayed 
the complete elimination of the NT computer and proprietary interface hardware 
(Sensable technologies’ I/O board and amplifier box) on the surgeon’s control station 
because we have found Sensable’s force generation capability useful for 
counterbalancing the surgeon’s control sticks (see above). Another project here at 
Berkeley has developed improved replacements for the proprietary equipment 
mentioned above. We intend to eventually incorporate these improvements into our 
surgeon’s control station. This will allow us to eliminate the NT computer after all, and 
still keep our counterbalancing feature. This approach will also leave open the 
possibility of eventually adding force feedback. 

Next-generation watchdog circuit. The next-generation watchdog will have the 
ability not only to detect a failure of the computer (as with the current one) but also to 
detect a successful start of the robot's computer and software. This takes 2 steps out of 
the startup procedure. Since the last report, expertise has been developed in the 
programming and use of gate array logic circuits. These will allow easy, reliable 
implementation of sophisticated watchdog algorithms. Additional uses of gate array 
logic in the project may include circuitry to maintain accurate encoder positions with 
minimal power, such as battery backup, even when the surgical robot is not plugged in. 
This could allow the robot to be prepped outside the operating room, and then wheeled 
in and set up quickly. 

Evaluate a 6 degree of freedom 3-dimensional mouse. A 6 D.O.F. haptic mouse 
may provide a more natural user interface for the robot than the existing haptic device 
which provides force feedback in only 3 of its degrees of freedom. Sensable 
Technologies has now developed a 6 D.O.F. mouse with full 6 D.O.F. force feedback. It 
may have sufficient resolution to replace the existing Phantom™ user interface device. 
We are considering whether this is something to which we wish to commit additional 
resources. This choice will be informed by what others in the field are learning about 
this and other 6 D.O.F. force feedback devices. 

New surgeon’s control sticks. In response to comments of surgeons, as well as 
their actions when using the robot, we have developed yet another set of control sticks 
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(see right frame of figure). These new control handles are designed to be held by the 
surgeon in the same way they already hold manual tools, such as needle drivers 
scissors, and hemostats. Our hope is that surgeons will be able to perform better with 
more familiar interface controls than they have with the current joysticks (left frame). As 
with the current control sticks, the mechanical configuration (particularly the relationships 
between rotational degrees of freedom) of the control has been matched to that of the 
robot. Thus, the new control sticks retain the benefit of the previous upgrade, which 
eliminated the need for a singularity in the kinematics calculations. 

Add a disposable boot to the manipulator. We have had difficulty arriving at a 
satisfactory design for a disposable boot to protect the manipulator from contamination. 
A disposable boot to prevent contamination and fouling of the manipulator must continue 
to guard against contamination reliably even after experiencing the wear inherent in 
performing gripping tasks as well as insertion and removal through the trocar. Also, the 
boot must interfere with neither manipulator function nor with visualization of the 
manipulator or the surgery site. It must also provide for any additional functions 
assigned to the end effector (such as electrocautery). The robotic surgical system can 
be used without such a boot, but the addition of such a boot would reduce the time and 
effort required for achieving satisfactory cleaning and sterilization after each use. We 
are considering design changes to the robot wrist and end effector that will enable fitting 
with a satisfactory protective boot. The design of the boot has been limited by the need 
to pass through the trocar without breaking. This may change for beating heart surgery 
where it is enough to avoid stopping the heart. In this case, having the tool be minimally 
invasive may not be necessary. Boot development has been delayed until we have a 
better idea what the full requirements will be. 

Add capability to adjust angle of user interface to robot. Since the last report, 
the system software has been modified to add the capability to adjust the angular 
position of the user interface device to the robot. This will allow the user interface device 
to be repositioned relative to the point simulating the trocar. Thus, when the camera is 
repositioned (to one side of the gripper, for example), the surgeon will be able to 
reposition the user interface device so that the correspondence between hand motions 
and gripper motions remains natural (that is, the front-back, left-right, and up-down 
translation axes still correspond). 
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